-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve CI #27
Merged
Merged
Improve CI #27
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mwaskom
force-pushed
the
michael/add-ci
branch
from
February 5, 2024 23:15
fadca2b
to
255b2ec
Compare
mwaskom
force-pushed
the
michael/add-ci
branch
from
February 6, 2024 16:55
747a0db
to
231562f
Compare
I'm going to merge this and then start a new PR where I update axolotl, etc. There are some remaining issues with the configs (e.g., does flash attention really not work with mistral?) but I think it makes more sense to investigate them once we're using the latest versions of things. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This sets up CI to run on pull requests against this repo using all three configs that we distribute.
To make CI go (somewhat) fast and be economical, I am configuring it to use some different parameters compared to the base configs. See the new
ci/prep_for_ci.py
script but namely:sequence_len
so that we can useA100-40GB
instancesval_set_size
,eval_batch_size
,micro_batch_size
Having struggled a
bitlot to get the CI to run due to surprising interactions between these variables (especially the validation set sizes) I am somewhat reconsidering whether this is a good idea and if it would instead be preferable to eat slow/expensive CI for the sake of distributing configs that we "know" will work. I think my preferred order of operations is: